Item selection using enhanced control

ABSTRACT

An enhanced control, in which a guide line is defined relative to an object in a user interface, items aligned with the guide line are displayed without obscuring the object. A selected item is output based on receiving a selection of one of the displayed items.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Patent Application No.60/952,448, filed Jul. 27, 2007, which is incorporated herein byreference.

BACKGROUND

Field

The present disclosure generally relates to a user interface and,according to one particular implementation, a control or widget.

Description of the Related Art

With regard to user interfaces, a control (or widget) is an interfaceelement that a computer user interacts with, such as a window or a textbox. In some cases, a control (such as a virtual button) may have asimilar function and appearance to a physical counterpart of thatcontrol.

SUMMARY

The enhanced control described herein may be used to facilitate theselection of an item from amongst a group of items, such as theselection of a letter from amongst a group of letters that make up analphabet, or the selection of a function from amongst a number offunctions. By orienting or aligning these items along a guide line, theymay be displayed so as to not overlap, block, or otherwise obscure aportion of an object also displayed within the user interface, to allowfor intuitive selection of the items by a user.

Selection may occur by hovering a cursor over the item for a set periodof time, or by selecting a physical control such as a mouse button orkeyboard key while the cursor is positioned over the item, or throughany another approach. Selection or potential selection of the item maycause the item to change its appearance (i.e. to be ‘activated’ or‘highlighted’), thereby distinguishing items from one another along theguide line and reducing selection errors. The object that provides thebasis for the definition or positioning of the guide line may itselfinteract with the items, such as where movement of an input device orother user input is mapped to an arm motion of an avatar surrounded bythe items aligned along the guide line.

According to one general implementation, a computer-implemented processincludes defining a guide line relative to an object in a userinterface, and displaying items aligned with the guide line withoutobscuring the object. The process also includes outputting a selecteditem based on receiving a selection of one of the displayed items.

Implementations may include one or more of the following features. Forinstance, the selection may be a cursor-based user selection. A positionof the object in the user interface may be determined, where the guideline may be dynamically defined to surround at least a portion of thedetermined position. A second object may be detected in the userinterface, where the guide line may be dynamically defined on a side ofthe object opposite to the second object. A change in the determinedposition may be detected, and the guide line may be redefined relativeto the object based on the determined change.

In further examples, the guide line may be a straight, curved, circular,polygonal, or zigzag shaped guide line, where each item may be analphanumeric character, a symbol, a setting, or a name. Outputting theselected item may further include highlighting the selected item, suchas by changing a color, opacity or size of the selected item. Changingthe color, opacity or size of the selected item may further includechanging the color, opacity or size of the selected item to a firstdegree, and changing the color, opacity or size of items adjacent to theselected item to a second degree. Displaying the items may furtherinclude evenly distributing the items along the guide line. The objectmay be a blank region in the user interface, or an avatar. A startingpoint and an ending point of the guide line may be aligned horizontallyor vertically.

In additional examples, defining the guide line may further includedetermining a range of motion of a control portion of the avatar, anddefining the guide line within the range of motion of the controlportion, where the guide line may be defined along an outer edge of therange of motion of the control portion. A first or a second potentialcontrol portion of the avatar may be designated as a control portion,the designated control portion may be animated, and the control portiondesignation may be swapped from the first potential control portion tothe second potential control portion, or from the second potentialcontrol portion to the first potential control portion.

According to a further general implementation, a computer programproduct is tangibly embodied in a machine-readable medium. The computerprogram product includes instructions that, when read by a machine,operate to cause data processing apparatus to define a guide linerelative to an object in a user interface, to display, without obscuringthe object, items aligned with the guide line, and to output, based onreceiving a selection of one of the displayed items, the selected item.

According to an additional general implementation, a device includes aprocessor and a user interface. The processor defines a guide linerelative to an object in a user interface. The user interface displays,without obscuring the object, items aligned with the guide line, andoutputs, based on receiving a selection of one of the displayed items,the selected item.

This brief summary has been provided to enable a quick understanding ofvarious concepts and implementations described by this document. A morecomplete understanding can be obtained by reference to the followingdetailed description in connection with the attached drawings. It is tobe understood that other implementations may be utilized and changes maybe made.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual diagram of a user interface that uses anexemplary enhanced control.

FIG. 2 is a block diagram of a device used to implement an exemplaryenhanced control.

FIG. 3 is a flowchart of a process that involves the use of an exemplaryenhanced control.

FIG. 4 illustrates a user interface that includes an object and anexemplary enhanced control that further includes a guide line.

FIGS. 5 to 11B illustrate example guide line arrangements and examplerelationships between guide lines and objects.

FIGS. 12 to 14 illustrate the exemplary selection of an item alignedalong a guide line.

FIGS. 15 and 16 illustrate the determination of a base position along aguide line.

FIGS. 17 to 19 illustrate subsets of items in various states associatedwith the highlighting of subsets.

FIGS. 20 and 21 illustrate exemplary velocity and scaling functions,respectively.

FIGS. 22 and 23 illustrate the use of a scaling feature by an exemplaryenhanced control.

FIGS. 24 and 25 illustrate exemplary guide lines in which a subset ofitems is highlighted.

FIG. 26 depicts the activation of an item along a guide line.

FIGS. 27 to 30 illustrate exemplary item sets.

FIG. 31 illustrates an user interface that includes one or moreexemplary enhanced control.

FIG. 32 illustrates an example of the exterior appearance of a computingdevice.

FIG. 33 is a block diagram illustrating the internal architecture of acomputer shown in FIG. 32.

DETAILED DESCRIPTION

FIG. 1 is a conceptual diagram of a user interface 100 using anexemplary enhanced control 102, shown in states 104 a to 104 d. In state104 a, a cursor 105 travels leftwards from the right side of a userinterface 100 that includes (among other things) an avatar 106 and thecontrol 102 disposed above the avatar 106. The control 102 includes aguide line 109 (shown in state 104 a as a dashed line), and items 110 ato 110 z, which each represent a letter of the English alphabet.Although the items 110 a to 110 z are illustrated as representingletters, they may also represent other things, such as icons, functions,images, objects, symbols, applications, characters, or groups orclusters of like or dissimilar items.

In state 104 a, since the cursor 105 is outside of the range of motionof the avatar 106, the avatar 106 is depicted in a neutral or relaxedposition, with the arms 111 a and 111 b of the avatar 1 06 relaxedalongside the torso 112 of the avatar 106.

The avatar 106 may have a human-like appearance, and may be able tonavigate within a virtual world. Examples of avatars include theplayable characters in video games like WORLD OF WARCRAFT® by BLIZZARDENTERTAINMENT® and virtual worlds such as SECOND LIFE® by LINDENRESEARCH, INC®. A cursor, such as a mouse cursor, may be mapped to acontrol portion of the object, such as a hand of the avatar, mirroringor mimicking movements of a cursor with movements of the avatar.

An ‘item’ is intended to refer to any unit or element that a user maydesire to select from a set of units or elements, including but notlimited to units or elements that are similar or dissimilar to the item.The set may include as few as no items, or as many as hundreds,thousands or millions of items. In one straightforward example, an itemis an alphabetic character, and the set of items includes the twenty sixletters in the English alphabet, or fifty two case sensitive letters.Similarly, the set could be made up of or include symbol characters,such as the symbols !, @, #, $, %, {circumflex over ( )}, &, *, (, ),{,}, <, >, :, ;, ′, ″, ?, /, ˜, ', or other symbols that are nottypically accessible via a keyboard or keypad selection. Other setscould include names, such as personal names or place names, selectablefrom a list of names, icons, functions, or settings, such as mediasettings (e.g. ‘Play,’ ‘Volume Up,’ ‘Power Off,’ ‘Record Series,’ etc.).

In this regard, items may include the numbers (0 through 9); letters(e.g. letters A through Z of an English alphabet, or kana charactersfrom Japanese script); text symbols (e.g., space “ ”, hyphen “-”, period“.”); predefined or dynamically determined text phrases (e.g., “.com”,“.org”, “FAQ”, “Main Menu”, “Send”, “Accept Call”, “Play DVD”, “PowerOff”, etc.); names; titles; times; dates; operations, events, processes,or functions (e.g., “Save Game”, “Load Game”, “Start Application”,“Transmit Data”, “Media Player”, “Photo Viewer”, “Paint Brush”, “Email”,“Map”); configuration options (e.g., “640×480 resolution mode”, “800×600resolution mode”, “Expert Mode”, “Novice Mode”, “Training Mode”, “CheatMode”); or any other information or combination of information.

State 104 b occurs when the cursor 105, now illustrated as a plussymbol, travels over a region of the user interface 100 occupied by item110 z, and is thus in a mouseover condition with the control 102. Upondetecting the mouseover condition, the arm 111 b of the avatar 106 isanimated to appear as if the avatar 106 is pointing at or grabbing theitem 110 z. Furthermore, the item 110 z becomes highlighted when itssize is increased, as does adjacent item 110 y to a lesser extent. Themouseover condition over the item 110 z also causes the item 110 z to beoutput, such as by displaying the item 110 z in output region 114, or byusing a text-to-speech converter to pronounce item 110 z using aspeaker. The mouseover condition may thus place an associated item in apre-selected, activated or highlighted state, although the pre-selectionor highlighting of an item does not strictly require the item to changeits appearance.

The user interface 100 transitions to state 104 c when the cursor 105continues to travel along the general path defined by the guide line109, until the cursor hovers over item 110 s of the control 102. The arm111 b of the avatar 106 is animated, and follows the movement of thecursor 105. As the cursor 105 approaches items or crosses the guide line109, the approached items become highlighted by increasing size, and asthe cursor 105 travels away from a highlighted item, the item reducessize. When a physical control on the input device is selected (e.g.,when an onClick event occurs), the activated item 110 s associated withthe mouseover condition is output.

In state 104 d, the item 110 s has been selected. The cursor 105 istraveling away from the control 102, whose items have all returned totheir original, non-highlighted condition. Furthermore, the arm 111 b ofthe avatar 106 has returned to its original, relaxed position adjacentto the torso 112. The selected item 110 s is output, for example in theoutput region 114.

As shown in the various states, the enhanced control may be used tofacilitate the selection of a letter or letters from amongst a group ofletters that make up an alphabet. By orienting or aligning these itemsalong the guide line, they are displayed so as to not overlap, block, orotherwise obscure the avatar or other object, to thereby allow forintuitive selection of the items by a user.

Although the selection of a letter in state 104 c has been described asoccurring via a mousedown event, selection may also occur by hoveringthe cursor 105 over the letter for a predetermined period of time, or byselecting a keyboard key while a tabbed cursor is positioned over theletter. Selection (or potential selection) of the letter may cause theitem to become highlighted, thereby distinguishing letters from oneanother along the guide line. An avatar or other object that is used fordefining the position of the guide line may itself interact with theitems, as in the case where movement of the cursor 105 is mapped tomotion of the arm 111 b the avatar 106.

FIG. 2 is a block diagram of a device 200 used to implement an exemplaryenhanced control. Briefly, the device 200 includes, among other things,a user interface 201, a storage medium 202, an input unit 204, and aprocessor 205.

The user interface 201 is a mechanism for allowing a user to interactwith the device, or with applications invoked by the device. The userinterface 201 may effect both input and output, allowing a user tomanipulate the device or for the device to produce the effects of theuser's manipulation. The device 200 may utilize any type of userinterface 201, such as a graphical user interface (GUI), a voice userinterface, or a tactile or haptic user interface.

The user interface 201 may be configured to render a visual displayimage. For example, the user interface 201 may be a monitor, atelevision, a liquid crystal display (LCD), a plasma display device, aprojector with a projector screen, an auto-stereoscopic display, acathode ray tube (CRT) display, a digital light processing (DLP)display, or any other type of display device configured to render adisplay image. The user interface 201 may include one or more displaydevices. In some configurations, the user interface 201 may beconfigured to display images associated with an application, such asdisplay images generated by an application, including an object such asan avatar.

The storage medium 202 stores and records information or data, and maybe an optical storage medium, magnetic storage medium, flash memory, orany other storage medium type. The input unit 204 is a hardwaremechanism that transforms physical inputs into commands that may be usedby the device, such as a keyboard or keypad, mouse, trackball, touchpad,or joystick. In certain implementations, such as where the device 201uses a touch screen, the functionality of the user interface 201 and theinput unit 204 may be combined into a single, combined module or unit.

The input unit 204 may be a device used to capture images, either asstill photographs or a sequence of moving images. The input unit 204 mayuse the light of the visible spectrum or with other portions of theelectromagnetic spectrum, such as infrared. For example, the input unit204 may be a digital camera, a digital video camera, or any other typeof device configured to capture images. The input unit 204 may includeone or more cameras. In some examples, the input unit 204 may beconfigured to capture images of an object or user interacting with anapplication. For example, the input unit 204 may be configured tocapture images of a user or person physically gesticulating infree-space, or otherwise interacting with an application within thefield of view of the input unit 204.

The input unit 204 may be a stereo camera, a time-of-flight camera, orany other camera. For instance the input unit 204 may be an imagedetector capable of sampling a background image in order to detectmotions and, similarly, gestures of a user. The input unit 204 mayproduce a grayscale image, color image, or a distance image, such as astereo camera or time-of-flight camera capable of generating a distanceimage. A stereo camera may include two image sensors that acquire imagesat slightly different viewpoints, where a processor compares the imagesacquired from different viewpoints to calculate the distance of parts ofthe images. A time-of-flight camera may include an emitter thatgenerates a pulse of light, which may be infrared light, where the timethe pulse of light travels from the emitter to an object and back to asensor is measured to calculate the distance of parts of the images.

The device 200 is electrically connected, over a wireline or wirelesspathway, to the input unit 204 and the user interface 201, and isconfigured to control the operation of the processor 205 to provide forthe enhanced control. In one configuration, the device 200 uses theprocessor 205 or other control circuitry to execute an application thatprovides for the enhanced control. Specifically, the device 200 receiveinputs from the input unit 204 and processes the received inputs tocalculate the position of the enhanced control in relation to an objectin the user interface 201, and to position a cursor, avatar, or otherrepresentation of the user in the user interface 201.

In one example implementation, input occurs by using a camera to detectimages of a user performing gestures. For instance, a mobile phone canbe placed on a table and may be operable to generate images of a userusing a face-forward camera. Alternatively, the gesture may berecognized or detected using the input device 204, such as by detectinga “tilt left” gesture to move a representation left and to select anitem disposed on the left side of a control, or by detecting a “tiltforward and right” gesture to move a representation up and to the rightof a neutral position, to select an item disposed on an upper right sideof a control.

The input device 204 may thus be any type of module operable to detectan angular position of the device 200, such as a gyroscope,accelerometer, or a camera-based optical flow tracker. In this regard,image-based input may be supplemented with or replaced by tilt-sensorinput to perform functions or commands desired by a user. Put anotherway, detection of a user's gesture may occur without using a camera. Bymoving the device in the same kind of stroke pattern that is visualizedon the control on the user interface, the user is enabled to control thesame interface or application in a straightforward manner.

Although the device 200 has been described as a personal computer (PC),such a description is made merely for the sake of brevity, and otherimplementations or manifestations are also contemplated. For instance,the device 200 may be implemented as a set top box, a television, anultra-mobile personal computer (UMPC), a mobile internet device (MID), adigital picture frame (DPF), a portable media player (PMP), ageneral-purpose computer (e.g., a desktop computer, a workstation, or alaptop computer), a server, a gaming device or console, or any othertype of electronic device that includes a processor or other controlcircuitry configured to execute instructions, or any other apparatusthat includes a user interface.

FIG. 3 is a flowchart of a process 300 that involves the use of anenhanced control. Briefly, the process 300 includes defining a guideline relative to an object in a user interface, and displaying itemsaligned with the guide line without obscuring the object. The process300 also includes outputting a selected item based on receiving aselection of one of the displayed items.

In more detail, when the process 300 begins (S301), a guide line isdefined relative to an object in a user interface. The process 300 maybegin when the user manually indicates that the enhanced control shouldbe displayed, or the process 300 may begin automatically when it isdetermined that an item are items are to be input, such as where a useradvances to a text input field. Since, by increasing the size ofhighlighted items, the enhanced control may be used to display moreitems along the guide line that would ordinary fit within the userinterface if all of the items had equal dimensions, the process 300 mayalso begin after a determination is made that all the items cannot bedisplayed effectively within the user interface without using theenhanced control. In doing so, the enhanced control provides for thedisplay of a subset of the items at a sufficient or effective size topermit easy and reliable user selection of a particular item.

Referring ahead briefly, FIG. 4 illustrates a user interface 401 thatincludes an object 402 (depicted as a dashed circle), and a control 405that further includes a guide line 404. Although the guide line 404 isillustrated as a dashed line in FIG. 4, in other example implementationsthe guide line 404 is depicted as a solid line, or is not depicted atall.

The control 405 allows for the intuitive selection or output of items,such as letters 406, while at the same time allowing for the display ofa portion or the entirety of the object 402 in a region adjacent to,near, partially or completely bounded or surrounded by, or otherwisedefined by the guide line 404. When it is determined that the items areto be output or selected, such as when a user selects an input fieldthat calls for the entry of text data, the control is positioned in alocation in the user interface 401 that allows for the display of theitems 406 and the object 402, and is thus defined relative to, inrelation to, based on, or with regard to the object 402. Put anotherway, the position of the object 402 may represent an anchor position, inrelation to which the guide line 404 and other elements of the control405 may be oriented or aligned, or with respect to which the otherelements may be positioned.

In order to determine this anchor position, the object by which theguide line 404 is to be defined, and the position of that object, aredetermined. In some cases, the object may be predetermined orpredefined, such as where the control 405 is output in conjunction withor includes an avatar, knob, icon, list, table of data, graph of data,text entry field, another control or widget, or known blank region ofthe user interface 401. In other cases, the object is dynamicallydetermined at the time when the control 405 is to be output, such aswhere the user tabs to a text field and the text field is used as theobject, where a blank region of the user interface 401 is located basedon the current state of the user interface 401, where a biggest,predominant, most or least colorful object on the screen is dynamicallydetermined to be the object, or where an in-focus element, region orwindow is dynamically determined to be the object.

In these and other cases, the size, shape, position, boundaries or othercontexts of the object 402 are detected, and the guide line 404 isdefined in relationship to these detected contexts. For instance, theguide line 404 may be defined to have an overlapping or non-overlappingrelationship with the object or a portion of the object, a bisectingrelationship, a dividing relationship, a space constrained or limitedrelationship, or any other relationship, such as a relationship based onsize, shape, proportion, or an anatomical model of the object 402.

In summary, and among other things, the user interface 401 includes anobject 402, which may be a representation of the user, and items 406that make up a set of items. The items 406 may be displayed in a dynamicfashion, such that the items 406 are displayed in a size and positionthat permits the user to conveniently and reliably select each item.Since it may otherwise not have been possible to fit each of the items406 within the user interface 401 or aligned with the guide line 404 ina large size, a subset of the items 406 may be rendered in that largersize.

FIGS. 5 and 6 illustrate example guide line arrangements and examplerelationships between guide lines and objects. In FIG. 5, a guide line501 is straight, and is defined to overlap an object 502, or to divide atop third of the object 502, or to separate a head 504 of the objectfrom a torso 505 of the object 502. In FIG. 6, a guide line 601 iscurved, and is defined to peak a certain pixel distance above the object602, or to not obscure the object 602, or to be within a range of motionof the arms 604 (or other control portion) of the object 602.

The guide line may be defined with respect to the object so that itemsaligned with the guide line are within reach of the object. Asillustrated in FIG. 6, the curved guide line 601 forms an arc shape, sothat any individual item or subset of items can be reached by one of theextended arms 604 of the avatar 602. The radius of curvature of thecurved guide line 601 may be determined based on the length of the arms604 of the avatar 602, such that the position of the curved guide line601 corresponds to a natural wave of one of the arms 604 when one of thearms 604 is in an extended, overhead position.

Although the guide line has been described as being below items, where acursor moves above the guide line to select an item, other layouts andconfigurations are possible. For example, a guide line may be positionedabove the selectable items or to a side of the selectable items, and maybe horizontally, vertically or diagonally oriented.

In addition to the straight and curved guide lines illustrated in FIGS.5 and 6, the guide line may take on a zigzag, circular, polygonal, orany other shape. A starting point and an ending point of the guide linemay be aligned horizontally or vertically, or these points may not be inany type of alignment. A guide line may be circular and continuous,whereby a guide line has no defined starting or ending point (or theappearance of no starting or ending point), or where the starting pointand ending point are coincident. The guide line may itself be a movingor animated object, such that items aligned along the guide line areconstantly or intermittently moving or animated themselves.

Range of motion determinations may be made based upon complex anatomicalmodels, such as a model that mimics the bio-kinetic restrictions orabilities of a human body and applies these abilities to an avatar.Simpler rules-of-thumb may also be used, such an approach that estimatesthat an avatar's arm may reach a distance equal to a dimension of theavatar's head multiplied by a multiplier. In any case, the particularpositioning, orientation, alignment, or configuration of the guide linesare defined by some aspect of the object. As such, object 602 is said tobe surrounded by the guide line 601.

Based on the position of the object, the guide line may be dynamicallydefined to surround a portion of the determined position. If the object,or a portion of the object, lies within a circle, and the guide linedefines an arc that is any portion of that circle, then the guide linemay be said to surround the object. As shown in FIG. 6, for example, ifextended, the guide line 601 would form a circle that would surround amajority of the object 602, with the exception of a small regionrepresenting the fingers of the object 602, which would lie outside thecircle.

As the definition of the guide line may be dynamic, the guide-line maybe redefined in real-time or in near-real-time, such that the user mayperceive or observe the redefinition or relocation of the guide line.For instance, the guide line may initially be defined above the object,as shown in FIG. 6, upon detecting that no other objects are adjacent tothe object. As shown in FIG. 7, if a second object 701 enters the userinterface 702 on the left side of the controlling object 704, the guideline 705 may be redefined or otherwise repositioned to move away fromthe second object 701, or be dynamically defined on a side of thecontrol object 704 opposite to the second object 701. Such a feature mayadvantageously allow the enhanced control to be more efficientlyutilized in non-static user interfaces that have multiple movingobjects.

Defining the guide line may further include determining a range ofmotion of a control portion of the avatar, and defining the guide linewithin the range of motion of the control portion, where the guide linemay be defined along an outer edge of the range of motion of the controlportion. For instance, the radius of curvature of the verticallyoriented guide line 705 may be determined based on the length of the armof the avatar, such that the position of the vertically oriented guideline 705 corresponds to a natural wave of the arm of the avatar when thearm is in an extended position to a side.

In an further example, such as where the number of items is too large tobe associated with a single guide line, or where the user interface iscrowded with other elements, each item may in fact represent multipleitems or clusters of items, or the guide line may be divided into partsand my thus define discontinuous portions, or multiple guide lines maybe defined. In FIG. 8, since the user interface 800 includes an object801 that occupies nearly its entire vertical length, the guide line 802is defined with a discontinuous portion or region 804 (illustrated as adashed line), effectively dividing the guide line 802 into guide lineportions 802 a and 802 b.

In FIG. 9, since the number of items to be selected or output is largeand could possibly make display or selection of the items on a singleguide line difficult or counterintuitive, two guide lines 901 and 902are defined within the user interface. Although guide lines 901 and 902are illustrated as having the same general shape and being parallel,neither characteristic must apply. For instance, the guide line 901 maybe a zigzag shaped guide line that is generally aligned vertically alongthe left side of the object 904, and the guide line 902 may be ahexagonal guide line that is defined to overlap a torso of the object904 itself, or a circular guide line that completely surrounds theobject 904 on all sides.

A selection of an item on a first guide line may spawn the definition ofor display of items aligned with a second guide line, such as where thefirst guide line is used for the selection of an alphabetical character,where names (or other item types) that start with an alphabeticalcharacter displayed on the first guide line are displayed otherwiseoutput on the second guide line once the alphabetical character has beenselected.

Instead of defining the guide line relative to the object, the guideline may also be defined or positioned based on the bounds of the userinterface. For instance, a guide line may be defined to extend from oneedge (or a point near one edge) of a user interface to an opposite edge(or a point near the opposite edge). The guide line may be defined orpositioned based on the positions or other components of the userinterface, for example, relative to a desktop icon, a user interfacebutton, or an object within a virtual world. The guide line may besymmetrical or asymmetrical with regards to the boundaries of the userinterface.

Returning to FIG. 3, items aligned with the guide line are displayed,with or without obscuring the object (S304). Items may be aligned withthe guide line if a bottom, middle, right, left, center, or otherportion of each item is on the guide line, or is disposed parallel to apoint on the guide line that corresponds to the item. In FIG. 4, forexample, items 406 a and 406 z are each in alignment with the guide line404 since portions of the respective items are parallel to the guideline 404, even though the items 406 a and 406 z are not strictly inalignment with each other. When displayed, the aligned items 406 as awhole take on the general shape or look of the guide line 404 itself.FIG. 5 also illustrates items aligned with the guide line 501, althoughfirst item 506 a and last item 506 z are also generally in alignmentwith each other since the guide line 501 is straight.

As shown in the user interface 1000 of FIG. 10, in the case where thecenter or other internal point of each item is aligned with the guideline 1001, each item may be rotated around its center to give thedisplayed items a less uniform, or more randomized look. Thisrandomization may be further emphasized by offsetting the items aboveand below a guide line (or some other line parallel to the guide line),such that some items appear above the guide line and some items appearbelow the guide line, although the aggregated items are generally stillaligned with, or take on the general look, feel, orientation,arrangement or disposition defined by the guide line.

The items may or may not be evenly distributed along the guide line. Forinstance, the items may initially be evenly distributed, and thenunevenly redistributed when a cursor hovers over a particular item andthat item is re-sized. Alternatively, default or preferred items may bedistributed along the guide line to appear more prominent than otherless preferred items. Further, the spacing between items can be definedby a pattern or mathematical function, or may be completely randomized.

Since the object or at least a portion of the object is visible when theitems are displayed, the object is deemed to be non-obscured. Putanother way, by displaying the items without obscuring the object, theitems do not completely overlap or block the visibility of the object.Certain characteristics of the items, such as the transparency, color,or line thickness of the items can be altered (or selectively altered,for example where the items would otherwise overlap the object), to makethe object more or less visible.

The items may be displayed without obscuring the object at a first pointin time, such as a time where the items are initially displayed, and theitems may obscure, occlude, obfuscate, block or overlap the object or aportion of the object at a second point in time that is earlier or laterthan the first point in time. By way of example, FIGS. 5 and 6 eachillustrate items that are displayed without obscuring the respectiveobjects, since at least a portion of the respective objects is visiblein the user interface. In a further example, FIG. 9 illustrates itemsthat are displayed without obscuring the associated object, since noportion of any item overlaps, blocks or interferes with the visibilityof any portion of the associated object within the user interface.

Returning to FIG. 3, a selection of one of the displayed items isreceived (S305). In one implementation, a user interface may define aregion around each item, where an item is selected if a cursor orcontrol portion of an object is detected is within the item's associatedregion. The regions may be dynamically refined based upon userinteractions, for example to increase in size based on a mouseovercondition or upon detecting that a cursor has crossed the guide line, orto decrease the size of the region when a cursor is far away, ishighlighting a different item, or has re-crossed the guide line.

As described in more detail below, the selection of an item may occurvia several approaches, depending upon the type of input unit that theuser uses to effect the selection. For instance, the selection may be acursor-based user selection using a mouse, in which the user positionsor hovers a mouse's cursor over an item to be selected (causing amouseover event), presses down a mouse button (causing a mousedownevent), and releasing the mouse button (causing a mouseup event). Othermouse events, such as a click event, a double-click (dblclick) event, ora mouseover event alone may also be used to select an item using amouse.

Using a keyboard, a user may tab from one item to another, and selectanother keyboard key (e.g. a spacebar or the return key) to select ahighlighted item, or the user may begin typing characters to narrow downor identify a desired item to select. Using a video game controller orhandheld remote control, a user may press a directional pad to changewhich item is highlighted, and press a button to select the highlighteditem. Using a mobile device with a tilt sensor, the user may tilt thedevice left, right, up or down to move a cursor or other indicator left,right, up or down until a desired item is highlighted, and then press akey or shake the mobile device to register the selection. Using a touchscreen device, the user may directly touch the X and Y coordinates ofthe user interface where the desired item is output. Using a voiceinterface, the user may say commands such as “tab,” “left,” “right,”“select,” or other similar voice commands move a cursor between items,and select a desired item.

A user may directly control an avatar using a mouse or video gamecontroller. For example, using a video game controller, a user may movean analog control stick to move the avatar's arm, whereby the angle ofthe analog control stick is mapped to an angle of the avatar's arm. Auser may directly control an avatar using a motion capture device,whereby the avatar's arms mimic the user's real arm motions.

The enhanced control described herein is compatible with video basedcontrol systems. Specifically, a camera may detect an image, such as animage of the user, and portions of the image, motion within the image,or identified gestures from the image may be dynamically mapped to acursor, in real time or near real time. Although further description ofother input mechanisms, approaches or techniques are omitted for thesake of brevity, it suffices to say that an item may be automatically ormanually selected using any conceivable approach, technique ormechanism.

The object may be a representation of a user, such as an avatar. Bydisplaying a representation of the user in the user interface, trainingrequirements are reduced since the user may easily recognize theobject's position relative to the position of the items, and may quicklymove a position of a cursor or control portion of the avatar (such as ahand) to be coincident with the position of a desired item. In thisregard, the position of the cursor or control portion of the avatar isused to detect selections or ‘touches’ of items displayed in the userinterface.

In the case where the object is an avatar, potential control portions ofthe avatar may be designated as a control portion to be animated. Forinstance, a control portion may be one or more arms, legs, elbows,knees, hands, or fingers of the avatar; or the head or torso of theavatar, or bodily features of the avatar such as a nose, eyes, ears,belly button, neck, or hair; or clothing, accessories or otheraccoutrements of the avatar such as clothing, jewelry or other personalitems. As user inputs are received, the designated control portion isanimated so as to interact with, or to give the appearance ofinteraction with, the items.

In the case where the object has the appearance of a knob or switch, acontrol portion may be an arrow that extends from the body of theobject, and the object may rotate to point towards an item. In the casewhere the object includes a particle system effect, such as a simulationof flames, plasma, lightning, or liquid, particles may form an extensionthat points towards an item, such as a lightning bolt or a deformationof a water droplet, to give the appearance of interaction with an item.

In FIG. 6, for example, the object 602 is an avatar, and the arm 604 aof the avatar is designated as the control portion. As the user moves amouse left or right, the arm 604 a is animated to move left or right,respectively, and items are selected based on the proximity of an itemto the arm or, more particularly, a hand or fingers on the arm.Similarly, the hand 607 may also be designated as the control portion.In similar arrangements, movement of a mouse up or down may cause thearm 604 a or hand 605 to move up or down, such as in manner befitting ofhuman motion.

Since these up or down motions may move the arm 604 a or the hand 605away from the items, at certain points when the arm 604 a or the hand605 is not near to (or is not interacting with) an item or the guideline, the ostensible “selection” of an item will not cause the item tobe output. By defining the guide line so that the avatar is for the mostpart below the guide line, the selection of characters or items by theobject may occur by positioning the avatar's hands overhead of theavatar and above the guide line.

The designation of a potential control portion of an avatar as thecontrol portion may be dynamically swapped to another control portion.For instance, and again referring to FIG. 6, if the user moves a mouseto the right past a position associated with item 606 n, the controlportion may be swapped from left arm 604 a to right arm 604 b, enablingthe left arm 604 a to interact with and select all items to the left ofitem 606 n along the guide line 601, and enabling the right arm 604 b tointeract with and select all items to the right of item 606 o along theguide line 601. Such a feature may increase the visual pleasure of theuser experience because, instead of merely selecting items along theitems using a plain mouse cursor, the user appears to be controlling anavatar that reacts in a realistic and intuitive manner.

If swapping occurs and the left arm 604 a is no longer designated as thecontrol object, the left arm 604 a is animated to return to a neutral,resting, or relaxed position, such as a position along the side of thetorso of the object 602. Conversely, such a swapping would cause theright arm 604 b to be seamlessly animated to move from the relaxedposition along the side of the torso of the object 602 to positionpreviously occupied by the arm 604 a, or a position adjacent to thatposition. Continuing with this example, if the user were to quickly movethe mouse cursor back and forth between the space along the guide linebounded by items 606 n and 606 o, the avatar could be animated asalternatively flapping their arms from the side of the object 602 tooverhead the object 602, in a pleasing, humorous, or visuallystimulating manner.

In a further example, the eyes of an avatar may be designated as thecontrol object, and they may appear to stare at or follow items along aguide line that may surround the object. Other portions of the avatarthat are not designated as the control object may also be animated, forinstance to appear to be react to a highlighted item, or to position theavatar in a desired pose or a more realistic pose based on a highlighteditem. In one implementation, a mouth of an avatar may be animated toappear as if it is speaking or otherwise verbalizing a selected orhighlighted item, or the neck or head of an avatar may be animated to becraned, thereby increase the emphasis that the appearance that theavatar is straining to look at or consider a highlighted object.

In a similar manner, if multiple objects (e.g. multiple avatars) aredisplayed in the user interface, a second object may be swapped in as acontrolling object based on a user selection, upon proximity to ahighlighted letter or the enhanced control, or based on any otherfactor. Thus, the control portion or control object designation may beswapped from the first potential control portion or object to a secondpotential control portion or object, or from the second potentialcontrol portion or object to the first potential control portion orobject.

Returning to FIG. 3, the selected item is output (S306), and the process300 ends (S307). Outputting the selected item may further includedisplaying the selected item, outputting an electronic indicia of theselected item, or highlighting the selected item by changing a color,opacity or size of the selected item within the user interface. Theselected item or an indicia of the selected item may be output toanother device or a storage medium for later use.

In FIG. 4, the color, opacity or size of the selected item 406 r hasbeen changed to a first degree or extent, thereby making the selecteditem 406 r appear the largest out of all of the items displayed alongthe guide line 404, and the color, opacity or size of items 406 p and406 t, which are adjacent to and equidistant from selected item 406 r,have been changed to a second, lesser degree or extent. By also changingthe color, opacity, size or other characteristic of adjacent items, theuser can identify the position of a cursor or of a selected item moreeasily, increasing the intuitive nature and ease of use of the control.The color, opacity, size, or other characteristic of items may indicatethe cursor position even where no cursor image is explicitly rendered.

Since the enhanced control allows for the selection of a small number ofitems at a given point in time and most items thus remain unselected,the unselected items may be reduced in size to allow for a large numberof items to be displayed within the user interface. Increasing the sizeof certain items under consideration for selection, however, mayincrease the overall recognition or readability of the items underconsideration for selection, thereby increasing reliability of thecontrol. Put another way, in decreasing the size of unselected items, alarger number of selectable items can be presented to a user than couldotherwise be discretely selected or touched.

As described in more detail below, the selected item 406 r may be outputin output region 407 of the user interface 401, along with previouslyselected item 406 f, and forecasted items 409, which are dynamicallydetermined based on the currently selected and previously selected itemsusing forecasting or prediction heuristics.

The enhanced control provides for the on-screen selection of items froma set of items, such as the selection letters from the alphabet. Arepresentation of the user is displayed, such that the representationmay conveniently and reliably touch selectable items aligned along aguide line or arc dynamically positioned relative to a representation. Atouch may be determined where a part of the representation intersectswith an item's region along the guide line. When a portion of therepresentation enters the touch region, items near that touch region maybe zoomed-in or otherwise highlighted.

Although the guide line is described above as being defined in twodimensions relative to the object or the bounds of the user interface,the guide line may also be defined in three dimensions. For instance,and as illustrated in FIG. 11 A, the guide line 1101 may have a firstguide line component 1102 generally defined within the same X-Y plane asthe user interface, as well as a second guide line component 1104generally defined within the Z plane, appearing to extend orthogonallyin relation to the X-V plane of the user interface. Other planes or axismay be used.

Using a three dimensional guide line, the number of items that can bedisplayed with the control may be exponentially increased. For instance,in addition to selecting the item “0” on the guide line 1101 by movingthe arms of the avatar 1106 left and right, the user may move the armsforward or backwards to select other items, such as the character “Q”1107, which is displayed in the output region 1109 when selected. Bothguide line components 1102 and 1104 are defined within thethree-dimensional reach of the arms of the avatar 1106.

Furthermore, although the guide line has been described above asincluding discrete starting and ending points, in other implementationsthe guideline may not have starting and ending points, or may have theappearance of no starting and ending points, or the starting and endingpoints may be coincident. User interface 1150 in FIG. 11B, for example,includes a circular guide line 1151 for the selection of items 1152 a to1152 g. The circular guide line includes starting point 1155 and endingpoint 1156, however these points are coincident and not otherwisehighlighted or visible on the guide line 1151 over any other points.

Furthermore, and unlike some other exemplary controls, the items 1152 ato 1152 g each correspond to a function to be invoked by an application,as represented by an icon. For instance, when the avatar 1154 selectsthe globe icon 1152 a, a mapping application may be invoked. As such,the enhanced control described herein can be used for the selection ofcharacters as well as to invoke more complex functionality, in aninstinctive and visually pleasing manner. Other icons may representother functions, including media functions such as a volume up or downfunction, a send mail function, a disable control function, or an imageviewer function

FIGS. 12 to 16 illustrate the exemplary selection of a particular itemfrom a set of items 1201 aligned along a guide line 1202. In FIG. 12, acursor 1204, which is mapped to and depicted as a hand of an avatar,moves toward an item of interest 1201 r. In FIG. 13, the position of thecursor 1204 crosses the guide line 1202. The crossing of the guide line1202 may initiate a further detection process which selects oridentifies a subset 1301 of the items 1201 based on a distance betweeneach item and the cursor 1204.

The subset 1301 of the items 1201 are displayed in a larger size orscale of font, facilitating easier selection by the user. Selection ofthe subset 1301 of the items 1201 and the enlarging of the display ofthe subset 1301 of the items 1201 may occur in response to detectingthat the cursor 1204 has crossed the guide line 1202, or may occurirrespective of the position of the cursor 1204 relative to the guideline 1202. Upon detecting that the cursor 1204 crosses the guide line1201 at base position 1302 or that an item has been selected, userfeedback may be generated, including for instance sound, imagery, and/ortactile output such as a vibration.

The user selects the item of interest 1201 r in FIG. 14. Highlightingthe subset 1301 permits the user to select a general area containing theitem of interest 1201 r, to “zoom in” on items within that area, and toreliable and conveniently select the item of interest 1201 r, whichrepresents the letter “R.” Selection may occur using mouse events,keyboard or keypad strokes, or through many other approaches.

The location of the base position 1302, representing the position wherethe cursor 1204 has crossed the guide line 1201, may be detected in manyways. For instance, and as shown in FIG. 15, the location of the baseposition 1302 may be determined as a position on the guide line 1201nearest to the cursor 1204 observed at a time after the cursor 1204 isdetected above the guide line 1201, or nearest to a highlighted itemsuch as item 1201 r.

The base position may be detected using other approaches as well. Forinstance, the location of the base position 1302 may be detected as thecursor 1204 crosses the guide line 1201, or using positions of thecursor 1204 at times before and after the crossing of the guide line1201. FIG. 16, for instance, illustrates endpoint 1601, representing aposition of the cursor 1204 observed at a time prior to crossing theguide line 1202, and endpoint 1602, representing a position of thecursor 1204 observed at a time after crossing the guide line 1202. Thebase position may be determined as the intersection of the line segment1604 defined by the endpoints 1601 and 1602, and the guide line 1201.

Highlighting the subset 1301 of the items 1201 may include determining aposition along the guide line 1201 of the items that make up the subset1301. In some implementations, items positioned in proximity to the baseposition 1302 are selected to be part of the subset 1301 of the items1201 to be highlighted (e.g. displayed in a large size), such that itemsnear the base position 1302 remain in or near their originalunhighlighted positions, and items further away from the base position1302 move outwards to accommodate for the increase in size of the subset1301 of the items 1201.

Equation (1), below, may be used to determine the position of items thatare not within the subset 1301 after the subset 1301 has beenhighlighted.

$\begin{matrix}{{X_{i}^{\prime} = {X_{b} + {\left( {X_{i\;} - X_{b}} \right) \cdot \left( \frac{S_{i}^{\prime}}{S_{i}} \right)}}}} & (1)\end{matrix}$

In Equation (1), X_(i) represents the position of item i along the guideline, in the original state; X_(i) represents the position of item ialong the guide line, in the enlarged state; X_(b) represents the baseposition along the guide line; S_(i); represents the base size of item iin the original state; and S_(i) represents the size of item i in theenlarged state

FIG. 17 illustrates a subset of items in a first state 1701 prior to thehighlighting of the subset of items, and in a second state 1702 afterthe highlighting of the subset of items. ⋅For example, if a cursor 1704initially crosses the guide line 1705 under the item “S” (such that thebase position 1706 is coincident to the item “S”), the item “S” remainsin its original position and the item “R” is displaced by distance 1707towards the left relative to its original position. The second state1702 thus demonstrates the items' scaled size and position followinghighlighting.

FIG. 18 illustrates a subset of items in a first state 1801 prior to thehighlighting of the subset of items, and in a second state 1802 afterthe highlighting of the subset of items. For example, if the cursor 1804crosses the guide line 1805 under the item “Q” (such that the baseposition 1806 is coincident to the item “Q”), the item “Q” will remainin its original position and the item “R” is displaced distance 1807towards the right relative to its original position. The second state1802 thus demonstrates the scaled items, where an item display 701represents the items' scaled size and position following highlighting.Accordingly, the position of the cursor 1804 used to select a particularitem may be dependent on the position where the cursor 1804 initiallycrossed the guide line 1805.

FIG. 19 illustrates a subset of items in states 1901 to 1904 associatedwith the highlighting of the subset of item. Specifically, FIG. 19 showsan overview of the selection and highlighting of first through thirdsubsets 1906 to 1908 of items 1910. The second and third subsets 1907and 1908 may be positioned according to the position of a cursor 1911relative to first and second subsets 1906 and 1907, respectively.

In state 1901, the items 1910 reflect their original, unhighlighted sizeand position. In state 1902, the first subset 1906 of the items 1910 hasbeen selected and highlighted. In state 1903, the second subset 1907 ofthe items 1910 has been selected and highlighted. In state 1904, thethird subset 1908 of the items 1910 has been selected and highlighted.

In state 1902, the cursor 1911 initially crosses the guide line 1912under the item “S” (such that the base position 194 is coincident to theitem “S”), the item “S” remains in its original position and thesurrounding letters are displaced outwards from their originalpositions. Transitioning from state 1902 to state 1903, if the cursor1911 moves to right, the second subset 1907 of the items 1910 within adistance of the cursor 1911 is selected. In the state 1903, if thecursor 1911 moves along the guide line 1912 to be coincident to theenlarged item “T”, the item “T” remains in its enlarged position and theitem “V” is highlighted moved distance 1915 further towards the rightalong the guide line 1912.

If insufficient space exists on the guide line 1912 for certain items,such as items “W” through “Z,” the items are “pushed off” the end of theguide line, and are not displayed. Transitioning from the state 1903 tostate 1904, if the cursor 1911 continues to move further towards theright end of the guide line 1912, there may also be insufficient spaceto display additional items that are part of the second subset 1907, andthe third subset 1908 may be formed (as a subset of the second subset1907).

In order to select an item on the right portion of the second subset1907 such as the item “U,” or an item that has been “pushed off” the endof the guide lines such as the items “W” to “Z,” the user may re-crossthe guide line 1912 with the cursor 1911, and cross the guide line 1912a third time to establish a new base position nearer to the desireditem. Furthermore, instead of “pushing off” items to the right end ofthe guide line 1912, items on the left end of the guide line may be“pushed off” instead to accommodate for the display of items that wouldotherwise be “pushed off” of the right end. In some implementations,instead of “pushing off” items, items may be decreased in size accordingto the available space on a guide line so that all items are displayedon a guide line.

In other implementations, items “pushed off” the right end of the guideline may re-appear on the left end of the guide line. In implementationsutilizing a continuous (e.g. circular) guide line, items may be pushedaround the guide line. Therefore, in an example of a circular guide linewhere a user may move a cursor clockwise around a continuous guideline,items excluded from the current subset of items may flow clockwise at alesser angular velocity than the cursor (as they make room for items togrow as they are added to the subset). In this example, a cursor maymove multiple revolutions around a guide line for one revolution of anitem around the guide line.

Scrolling may be used to facilitate a selection of items that wouldotherwise have been pushed off an end of a guide line. Scrolling mayinclude detecting if a cursor 1911 is within a predefined distance of anend of a guide line 1912, and applying a velocity to item positions.Where item positions are calculated relative to a base position 1914(see Equation (1), above), the velocity may be applied to the baseposition 1914, and items may be displaced accordingly.

FIG. 20 illustrates an exemplary velocity function, in which thehorizontal axis 2001 represents positions along a guide line, and thevertical axis 2002 represents velocity. Using this velocity function,items are moved or shifted by applying a velocity when the cursorposition is near an end of the guide line, and items are not moved orshifted (since the velocity is zero) when the cursor position is at thecenter of the guide line.

FIG. 21 illustrates an exemplary scaling function which may be used toscale the size of an item (reflected by vertical axis 2001) in a subsetof items based on a distance (reflected by horizontal axis 2001) betweenthe position of a cursor relative to a guide line. As represented by thecurve 2004, the size of an item (S₁) may thus be a function of thecurrent position of a cursor. A base size (S₁) associated withunhighlighted items or all items if the cursor has not crossed the guideline is determined by referencing point 2005, and line 2006 defines amaximum size associated with a highlighted item.

In one example implementation, the slope of the curve 2004 isapproximately 1:1 near the point 2005, so that item sizes appears togrow linearly and proportionally as the proximity to the cursordecreases. Growth of the item would begin to off near the maximum size,to produce an aesthetically pleasing transition as the cursor becomesincreasingly proximate to a highlighted item.

FIGS. 22 and 23 illustrate the use of the scaling feature by theenhanced control, specifically occurring when a cursor 2201 crosses aguide line 2202. A height 2204 represents a distance between theposition of the cursor 2201 and the guide line 2202, and a height 2205represents a size of items, such as item 2206 r, in a subset 2207. Theheight 2205 of items in the subset 2207 is scaled based on the height2204, making items appear larger in FIG. 22, where the cursor 2201 hascrossed the guide line 2202 by a small amount, than in FIG. 23, wherethe cursor 2201 has crossed the guide line 2202 by a smaller amount.

FIGS. 24 and 25 illustrate exemplary guide lines in which a subset ofitems is highlighted. Specifically, FIG. 24 illustrates an exemplaryguide line 2401 in which a subset 2402 of items 2404 is displayed in anenlarged manner. Selecting the subset 2402 of items 2404 may includeselecting a predefined number of items, or dynamically selecting anumber of items to include within the subset 2402 based on the size ofthe items.

The number of items to include in the subset 2402 may be dynamicallyselected so that the items of subset 2402 span the entire length of theguide line 2401, or they may span a portion of the guide line 2401. Asshown in FIG. 24, items 2405 a and 2405 b that are excluded from thesubset 2402 may also be displayed along the guide line 2401. The numberof items in the subset 2402 may also change based on size of the itemswithin the subset, for example where subset 2207 (in FIG. 22) displaysfive items, and subset 2207 (in FIG. 23) displays three items, eventhough the width of the subset 2207 remains the same.

Highlighting items by displaying the items of a subset at a large sizemay include displaying all of the highlighted items at the same largesize, as illustrated in FIG. 24, or by displaying individual itemswithin a subset at a size that is dependent on its position along theguide line relative to a cursor position, as illustrated in FIG. 25.

FIG. 25 illustrates an exemplary guide line 2501 in which a subset 2502of items 2504 is displayed with a varying item size. For example, thesize of items 2505 p and 2505 t at the ends of the subset 2502(representing letters “P” and “T,” respectively), may be sized smallerthan the item or items at the center of the subset 2502, such as theitem 2505 r (representing the letter “R”). Displaying items in thesubset 2502 with varying sizes may produce a pleasing aestheticappearance and may make use of the enhanced control more intuitive.

Displaying the items of the subsets 2402 and 2502 at a large size mayinclude animating the items. An animation may include enlarging theitems of the subset and translating the position of items along theguide line (e.g., keeping the items perpendicular to the guide line)over a short period of time. Items that are excluded from the subsets2402 and 2502 may be animated to shrink in size and move outwards alongthe guide line, in order to “make room” for the subset 104.

Items that are “pushed off” from the ends of a guide line may simplydisappear, or may be animated to fall off the edge of the guide line orto be destroyed in a visually stimulating or humorous manner, such as bycombustion, implosion, vaporization, explosion, liquefaction, crushing,or other techniques. Similarly, previously “pushed off” items thatreappear due to space being made on the guide line may simply re-appear,or may be animated to fall from the top of the user interface back ontothe guide line or to be spontaneously generated in a visuallystimulating or humorous manner.

Returning to FIG. 8, where a guide line is discontinuous, items may beanimated to move across the discontinuity. Items may be animated to moveacross the gap at a high velocity, or may be animated to be “pushed off”and “reappear” using any of the above visual effects. Likewise, itemsthat are “pushed off” one end of a guide line to re-appear on theopposite end may be animated to move between the guide line end pointsat a high velocity, or may be animated to be “pushed off” and “reappear”using any of the above visual effects.

FIG. 26 depicts the activation of an item along a guide line, where“activation” or “highlighting” generally relate to the determination ofan item to select. A system to determine an item to select may usehysteresis. Selection may include determining an initial selected itemwhen a cursor initially crosses a guide line, where the initial selecteditem may be the item whose position is nearest in distance to the baseposition. Thereafter, in order to reduce inadvertent flicker between twoadjacent items when a cursor is disposed in between items, selection mayinclude determining a new selected item whenever the cursor moves apredefined distance beyond the midpoint between adjacent items.

For example, the position along a guide line 2601 of two items 2602 rand 2602 s are indicated by lines 2604 r and 2604 s, the position of themidpoint between the items 2602 r and 26025 is indicated by line 2605,and a distance 2606 represents a predefined distance. If, for example,the item 2602 s (i.e. the letter “S”) is the initial selected item, theuser would move a cursor left of line 2607, which is the predefineddistance 2606 beyond the midpoint 2605 in the direction towards the item2602 r (i.e. the letter “R”) to select the item 2602 r. If the item 2602r is subsequently selected, to re-select the item 2602 s, the user wouldmove a cursor right of line 2609, which is the predefined distance 2606beyond the midpoint 2605 in the direction towards the item 2602 s.

The predefined distance may be defined based upon a desired maximumdistance a cursor may be expected to waver or shake while the userattempts to remain still, accounting for the effects of user movementscaused by involuntary body tremor or limitations of the input unit Anitem may be deselected when the position of the cursor is below theguide line, or when a cursor is not found or is not within a particularuser interface or region of the user interface occupied by the enhancedcontrol. Selection of an item may further generate user feedback,including for example sounds, imagery, and/or a tactile output such as avibration.

Displaying the items of the subset may include displaying items suchthat their appearance provides user feedback as to their state ofselection. For example, a selected item may be displayed in a uniquecolor, or be displayed with a visual effect such as the appearance ofglowing. A currently selected item may be activated or highlighted whenthe item is in a selected state for longer than a first predefinedduration threshold. In this regard, an item is activated when a cursoris held over an item for a period of time. Activation may be repeated ornegated if a selected item remains selected for longer than a secondpredefined duration threshold.

Moreover, an item may be activated or highlight when a position of acursor is stationary for a period of time. A cursor may be classified asstationary when the change in the component of position parallel to theguide line is less than a predefined distance threshold for more than apredefined time threshold. For example, an item closest to the positionof the cursor is identified and activated. Activation may be repeated ornegated if a cursor remains classified as stationary for longer than asecond predefined duration threshold. Additionally, an item may beactivated based on a distance between a cursor position and relative aguide line. For instance, an item may be activated when such a distanceexceeds a predefined distance threshold 803, as shown in FIGS. 22 and23.

In other examples, activation of item may result from another type ofuser input after selection of the item. For instance, the user mayprovide another type of user input to activate a selected item. In theseexamples, to activate a selected item, the user may touch a user inputbutton (e.g., on a controller), provide an audible input (e.g., saying“activate”), performing another type of gesture (e.g., moving the handused to select the item toward the display or moving the user's otherhand to another portion of the display image to activates the selecteditem), or providing any other type of user input.

FIGS. 27 to 30 illustrate exemplary item sets. In FIG. 27, items 2701,which include characters from the English alphabet, includes an item2702 that, when activated or otherwise selected, opens a second set ofitems. When selected, the second set of items may appear along the guideline 2704 with the items 2701 or a portion of the items 2701, or thesecond set of items may replace the items 2701 on the guide line 2704.From the symbol used to represent the item 2702, the user mayintuitively determine that the second set of items includes numbers.

FIG. 28 illustrates items 2801 aligned with a guide line 2802. Items2801 are displayed when the item 2702 in FIG. 27 is selected. Once thesecond set of items is selected, removal of the cursor below the guideline 2802 or removing the cursor from the user interface that displaysthe items 2801 may cause the items 2701 to be reselected orre-activated. The items 2801 includes an item 2804 that, when activated,re-opens, re-activates or otherwise re-selects the items 2701. From thesymbol used to represent the item 2804, the use may intuitivelydetermine that the items 2701 include characters from the Englishalphabet.

FIG. 29 illustrates items 2901 aligned with a guide line 2901, where theitems each represent a combination of a number and letters that areknown to be associated with keys on a standard telephone phone keypad.FIG. 30 illustrates items 3001, which are displayed when item 2902 ofitems 2901 are selected (see FIG. 29), and which include the combinationof characters the number “7” associated with the key number “7” on astandard telephone keypad.

Using the items illustrated in FIGS. 27 to 30, a text input mechanismmay be provided, such as by sequentially filling in letters that formwords into a text field. Upon activating a number, letter, text symbol,or predefined text, the number, letter, text symbol, or predefined textmay be appended to a string of characters that has already been input.Alternatively, activation of an item (such as an item from items 2701 inFIG. 27) may display further items, and activation of an item of thefurther items may appends the activated character onto a string ofcharacters. One of the items may be a backspace item, to facilitateremoval of the latest item from the string of items.

A text input mechanism may include combining characters to form compoundcharacters. A text input mechanism for Japanese text input may includecombining kana characters to form kanji characters (in a method familiarto users of Japanese personal computer keyboards). A first set of itemsmay include items representing kana. When a kana item is activated, thecorresponding kana character is appended to a string of characters. Aset of items may include an item representing an operation, that whenactivated, activates a process that converts the latest kana charactersof the string into kanji. The process of converting the latest kanacharacters of the string into kanji may include displaying a second setof candidate kanji items. Activating a kanji item activates a processwhere the latest kana characters of the string are replaced with theactivated kanji. This text input method may be extended to otherlanguages.

An alternative method for Japanese text input may include displaying afirst set of items representing kana. When a kana item is activated, asecond set of items is displayed. The second set of items may includethe kanji for which the activated kana forms a part of. This set may belarger than the second set described above. This text input method maybe extended to other languages.

A text input mechanism may include a confirmation item in a set ofitems. When activated, a confirmation item activates a process wherebythe string of characters is provided to an application. Furthermore, thetext input mechanism may include a predictive text completion process. Apredictive text completion process may search a dictionary to find themost likely text that contains the characters of the string ofcharacters. The most likely text may be displayed in an output field. Adictionary used in the predictive text completion process may beselected based on the context in which the text input mechanism is used.For example, a dictionary may include names when the text inputmechanism is used to enter a name.

An application process may determine items for the user interface,depending on the state of the application. Activating an item mayprovide a message to an application process. The application process maybe controlled based on the message.

FIG. 31 illustrates an exemplary user interface 3100 including anexemplary enhanced control. As the user accesses the various text entryfields on the user interface 3100, the text entry fields are used asobjects to define guide lines, and items appropriate to the text entryfields are displayed aligned with the guide lines without obscuring thetext entry fields themselves. The user may select the items aligned withthe guide lines, to populate the associated text entry fields. Althoughthe user interface 3100 appears to illustrate multiple visible enhancedcontrols at once, such a simultaneous display is merely exemplary and isdone to illustrate possible control positions, configurations,alignments and item types. In other user interfaces, for example, oneenhanced control would be displayed at a time.

For instance, if the user accesses text entry field 3101, such as bytabbing to that field or selecting that field with a mouse cursor, guideline 3102 is defined relative to the text entry field 3101, and items3104 are displayed aligned with the guide line 3102 so as to not obscurethe text entry field 3101. Since the text entry field 3101 accepts textor character data, the enhanced control automatically determines thatalphabetical character items 3104 are appropriate for the object type.When a user selects an item from the items 3104, the text entry field3101 is populated with the selected item. Instead of selecting itemsfrom the guide line 3102, the user may also use the enhanced control3105, defined around avatar 3106, for populating the various fields withitems.

As the user tabs to or otherwise accesses other fields within the userinterface 3100, other enhanced controls may be dynamically defined,outputting items for selection. For instance, accessing social securitynumber field 3107 may cause numerical items 3109 to be displayed abovethe field 3107; accessing gender field 3110 may cause gender items 3111to be dynamically output above the field 3110; accessing citizenshipfield 3112 may cause country items 3113 to be displayed on two guidelines 3114 and 3115 above and below the field 3112, respectively;accessing marital status field 3117 may cause marital status indicatoritems 3119 to be displayed on a guide line 3110 on the right side of thefield 3117, where the guide line 3110 may be dynamically defined on theright side due to space constraints, user preferences, or other reasons;accessing street address field 3120 may cause items 3121 that includenumbers and an alphabetical symbol 3122 that replaces the numbers withalphabetical characters along a guide line 3124 to be displayed on theguide line 3124 above the field 3120; and accessing state field 3125 maycause items 3126 that include state names to be displayed on two guidelines 3127 and 3129 defined above the field 3125.

FIG. 32 illustrates an example of the exterior appearance of a computingdevice 3201 that further includes a processor and a user interface. Theprocessor defines a guide line relative to an object in a userinterface. The user interface displays, without obscuring the object,items aligned with the guide line, and outputs, based on receiving aselection of one of the displayed items, the selected item.

In more detail, the hardware environment of the computing device 3201includes a display monitor 3208 for displaying text and images tointerface with a user, a keyboard 3209 for entering text data and usercommands into the computing device 3201, a mouse 3210 for pointing,selecting and manipulating objects displayed on the display monitor3208, a fixed disk drive 3211, a removable disk drive 3212, a tape drive3214, a hardcopy output device, a computer network connection, and adigital input device 3217.

The display monitor 3208 displays the graphics, images, and text thatcomprise the user interface for the software applications used by thecomputing device 3201, as well as the operating system programsnecessary to operate the computing device 3201. A user uses the keyboard3209 to enter commands and data to operate and control the computeroperating system programs as well as the application programs. The mouse3210 may be any type of pointing device, and may be a joystick, atrackball, a touch-pad, or other pointing device. Software used todisplay a user interface and enable a user to enter or select text,numbers, or select from a menu of options is stored locally on computerreadable memory media, such as the fixed disk drive 3211.

In a further implementation, the fixed disk drive 3211 itself mayinclude a number of physical drive units, such as a redundant array ofindependent disks (“RAID”), or may be a disk drive farm or a disk arraythat is physically located in a separate computing unit. Such computerreadable memory media allow the computing device 3201 to accesscomputer-executable process steps, application programs and the like,stored on removable and non-removable memory media.

The computer network connection may be a modem connection, a local-areanetwork (“LAN”) connection including the Ethernet, or a broadbandwide-area network (“WAN”) connection such as a digital subscriber line(“DSL”), cable high-speed internet connection, dial-up connection, T-1line, T-3 line, fiber optic connection, a broadband over power lineconnection, or satellite connection. The network 1306 may be a LANnetwork, a corporate or government WAN network, the Internet, or othernetwork.

The computer network connection may be a wireline or wireless connector.Example wireless connectors include, for example, an INFRARED DATAASSOCIATION® (“IrDA®”) wireless connector, an optical wirelessconnector, an INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS®(“IEEE®”) Standard 802.11 wireless connector, a BLUETOOTH® wirelessconnector, an orthogonal frequency division multiplexing (“OFDM”) ultrawide band (“UWB”) wireless connector, a time-modulated ultra wide band(“TM-UWB”) wireless connector, or other wireless connector. Examplewired connectors include, for example, a IEEE®-1394 FIREWIRE® connector,a Universal Serial Bus (“USB”) connector, a serial port connector, aparallel port connector, or other wireline connector.

The removable disk drive 3212 is a removable storage device that is usedto off-load data from the computing device 3201 or upload data onto thecomputing device 3201. The removable disk drive 3212 may be a floppydisk drive, an IOMEGA® ZIP® drive, a compact disk-read only memory(“CD-ROM”) drive, a CO-Recordable drive (“CD-R”), a CD-Rewritable drive(“CD-RW”), flash memory, a USB flash drive, thumb drive, pen drive, keydrive, a High-Density Digital Versatile Disc (“HD-DVD”) optical discdrive, a Blu-Ray optical disc drive, a Holographic Digital Data Storage(“HDDS”) optical disc drive, or any one of the various recordable orrewritable digital versatile disc (“DVD”) drives such as theDVD-Recordable (“DVD-R” or “DVD+R”), DVD-Rewritable (“DVD-RW” or“DVD+RW”), or DVD-RAM. Operating system programs, applications, andvarious data files, are stored on disks, which are stored on the fixeddisk drive 3211 or on removable media for the removable disk drive 3212.

The tape drive 3214 is a tape storage device that is used to off-loaddata from the computing device 3201 or to upload data onto the computingdevice 3201. The tape drive 3214 may be a quarter-inch cartridge(“QIC”), 4 mm digital audio tape (“DAT”), 8 mm digital linear tape(“DLT”) drive, or other type of tape.

Furthermore, although the computing device 3201 is described above as adesktop PC, in further implementations the computing device 3201 may bea laptop, a workstation, a midrange computer, a mainframe, an embeddedsystem, telephone, a handheld or tablet computer, a PDA, a gaming deviceor console, a digital picture frame, a teleconferencing device, or othertype of computer.

FIG. 33 is a block diagram illustrating the internal architecture of acomputer shown in FIG. 32. An exemplary internal architecture of thecomputing device 3201 is now described. The computing environmentincludes a computer central processing unit (“CPU”) 3301, where thecomputer instructions that comprise an operating system or anapplication are processed; a display interface 3302 which provides acommunication interface and processing functions for rendering graphics,images, and texts on the display monitor 3208; a keyboard interface 3304which provides a communication interface to the keyboard 3209; apointing device interface 3305 which provides a communication interfaceto the mouse 3210 or an equivalent pointing device; a digital inputinterface 3306 which provides a communication interface to the digitalinput device 3217; a hardcopy output device interface which provides acommunication interface to the hardcopy output device; a random accessmemory (“RAM”) 3310 where computer instructions and data are stored in avolatile memory device for processing by the computer CPU 3301; aread-only memory (“ROM”) 3311 where invariant low-level systems code ordata for basic system functions such as basic input and output (“1/0”),startup, or reception of keystrokes from the keyboard 3209 are stored ina non-volatile memory device; and a storage 3320 or other suitable typeof memory (e.g., such as random-access memory (“RAM”), read-only memory(“ROM”), programmable read-only memory (“PROM”), erasable programmableread-only memory (“EPROM”), electrically erasable programmable read-onlymemory (“EEPROM”), magnetic disks, optical disks, floppy disks, harddisks, removable cartridges, flash drives), where the files thatcomprise an operating system 3321, application programs 3322 (includingthe enhanced control application 3323, and other applications 3324 asnecessary) and data files 3325 are stored; a computer network interfacewhich provides a communication interface to the network over thecomputer network connection. The constituent devices and the computerCPU 3301 communicate with each other over the computer bus 3327.

According to one general implementation, a computer program product istangibly embodied or recorded in a machine-readable medium such asstorage 3320. The computer program product includes instructions that,when read by a machine, operate to cause data processing apparatus todefine a guide line relative to an object in a user interface, todisplay, without obscuring the object, items aligned with the guideline, and to output, based on receiving a selection of one of thedisplayed items, the selected item.

The RAM 3310 interfaces with the computer bus 3327 so as to providequick RAM storage to the computer CPU 3301 during the execution ofsoftware programs such as the operating system application programs, anddevice drivers. More specifically, the computer CPU 3301 loadscomputer-executable process steps from the fixed disk drive 3211 orother memory media into a field of the RAM 3310 in order to executesoftware programs. Data is stored in the RAM 3310, where the data isaccessed by the computer CPU 3301 during execution.

The computing device 3201 stores computer-executable code for anoperating system 3321, application programs 3322 such as wordprocessing, spreadsheet, presentation, gaming, or other applications.Although it is possible to output the enhanced control a user interfaceand enable a user to enter text, numbers, or select from the controlusing the above-described implementation, it is also possible toimplement the functions according to the present disclosure as a dynamiclink library (“DLL”), or as a plug-in to other application programs suchas an Internet web-browser such as the MICROSOFT® Internet Explorer webbrowser.

The computer CPU 3301 is one of a number of high-performance computerprocessors, including an INTEL® or AMD® processor, a POWER PC®processor, a MIPS® reduced instruction set computer (“RISC”) processor,a SPARC® processor, an ACORN® RISC Machine (“ARM®”) architectureprocessor, a HP ALPHASERVER® processor or a proprietary computerprocessor for a mainframe. In an additional arrangement, the computerCPU 3301 is more than one processing unit, including a multiple CPUconfiguration found in high-performance workstations and servers, or amultiple scalable processing unit found in mainframes.

The operating system 3321 may be MICROSOFT® WINDOWS NT®/WINDOWS®2000/WINDOWS® XP Workstation; WINDOWS NT®/WINDOWS® 2000/WINDOWS® XPServer; a variety of UNIX®-flavored operating systems, including AIX®for IBM® workstations and servers, SUNOS® for SUN® workstations andservers, LINUX® for INTEL® CPU-based workstations and servers, HP UXWORKLOAD MANAGER® for HP® workstations and servers, IRIX® for SGI®workstations and servers, VAX/VMS for Digital Equipment Corporationcomputers, OPENVMS® for HP ALPHASERVER®⋅based computers, MAC OS® X forPOWERPC® based workstations and servers; SYMBIAN OS®, WINDOWS MOBILE® orWINDOWS CE®, PALM®, NOKIA® OS (“NOS”), OSE®, or EPOC® for mobiledevices, or a proprietary operating system for computers or embeddedsystems. The application development platform or framework for theoperating system 3321 may be: BINARY RUNTIME ENVIRONMENT FOR WIRELESS®(“BREW®”); Java Platform, Micro Edition (“Java ME”) or Java 2 Platform,Micro Edition (“J2ME®”); PYTHON™, FLASH LITE®, or MICROSOFT®.NETCompact.

While FIGS. 32 and 33 illustrate one possible implementation of acomputing device that executes program code, or program or processsteps, configured to provide for an enhanced control that allows for auser to intuitively and easily enter text, numbers, or select from aplurality of items, other types of computers or implementations may alsobe used as well.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A computer-implemented method comprising:determining, after an object is displayed in a user interface, aposition of a non-control portion of the object in the user interface,wherein the object comprises: a control portion used for item selection,and the non-control portion; responsive to the determining of theposition of the non-control portion of the object in the user interface,defining a guide line at a location within the user interface based on:the determined position of the non-control portion of the object, and adetermined range of motion of the control portion of the object:displaying, without overlapping with the non-control portion of theobject, a plurality of items aligned with the guide line, wherein thedetermining the position of the non-control portion of the object in theuser interface occurs before the plurality of items are displayed;highlighting, without moving the guide line, a subset of the pluralityof items; receiving a selection of one item of the subset; andoutputting, based on receiving the selection of one item of the subset,the selected item.
 2. The computer-implemented method recited in claim1, wherein highlighting the subset of the plurality of items is based ona location of the control portion of the object.
 3. Thecomputer-implemented method recited in claim 1, wherein highlighting thesubset of the plurality of items comprises increasing a size of eachitem of the subset.
 4. The computer-implemented method recited in claim3, further comprising repositioning one or more items of the pluralityof items to accommodate the increase in size of each item of the subset.5. The computer-implemented method recited in claim 1 wherein the userinterface is shown on a display of a mobile device.
 6. Thecomputer-implemented method recited in claim 1 wherein determining theposition of the non-control portion of the object in the user interfaceis based on an image of a user captured by a camera of a mobile device.7. The computer-implemented method recited in claim 1 wherein theselection of one item of the subset is based on input received by atouch screen.
 8. A device comprising: a memory; and a processor coupledwith the memory and configured to cause the device to: determine, afteran object is displayed in a user interface, a position of a non-controlportion of the object in the user interface, wherein the objectcomprises: a control portion used for item selection, and thenon-control portion; responsive to the determining of the position ofthe non-control portion of the object in the user interface, define aguide line at a location within the user interface based on: thedetermined position of the non-control portion of the object; and adetermined range of motion of the control portion of the object;display, without overlapping with the non-control portion of the object,a plurality of items aligned with the guide line, wherein thedetermining the position of the non-control portion of the object in theuser interface occurs before the plurality of items are displayed;highlight, without moving the guide line, a subset of the plurality ofitems; receive a selection of one item of the subset; and output, basedon receiving the selection of one item of the subset, the selected item.9. The device recited in claim 8, wherein the processor is furtherconfigured to cause the device to highlight the subset of the pluralityof items based on a location of the control portion of the object. 10.The device recited in claim 8, wherein the processor is furtherconfigured to cause the device to highlight the subset of the pluralityof items by increasing a size of each item of the subset.
 11. The devicerecited in claim 10, wherein the processor is further configured tocause the device to reposition one or more items of the plurality ofitems to accommodate the increase in size of each item of the subset.12. The device recited in claim 10, further comprising a display,wherein the processor is further configured to cause the device to showthe user interface on the display.
 13. The device recited in claim 10,further comprising a camera, wherein the processor is further configuredto cause the device to determine the position of the non-control portionof the object in the user interface based on an image of a user capturedby the camera.
 14. The device recited in claim 10 wherein the selectionof one item of the subset is based on input received by a touch screen.15. An apparatus comprising: means for determining, after an object isdisplayed in a user interface, a position of a non-control portion ofthe object in the user interface, wherein the object comprises: acontrol portion used for item selection, and the non-control portion;means for, responsive to the determining of the position of thenon-control portion of the object in the user interface, defining aguide line at a location within the user interface based on: thedetermined position of the non-control portion of the object, and adetermined range of motion of the control portion of the object; meansfor displaying, without overlapping with the non-control portion of theobject, a plurality of items aligned with the guide line, wherein thedetermining the position of the noncontrol portion of the object in theuser interface occurs before the plurality of items are displayed; meansfor highlighting, without moving the guide line, a subset of theplurality of items; means for receiving a selection of one item of thesubset; and means for outputting, based on receiving the selection ofone item of the subset, the selected item.
 16. The apparatus recited inclaim 15, wherein the means for highlighting the subset of the pluralityof items include means for basing the highlighting on a location of thecontrol portion of the object.
 17. The apparatus recited in claim 15,wherein the means for highlighting the subset of the plurality of itemsinclude means for increasing a size of each item of the subset.
 18. Theapparatus recited in claim 17, further comprising means forrepositioning one or more items of the plurality of items to accommodatethe increase in size of each item of the subset.
 19. The apparatusrecited in claim 15, wherein the apparatus comprises a mobile device.20. The apparatus recited in claim 15, further comprisingimage-capturing means, wherein the means for determining the position ofthe non-control portion of the object in the user interface includemeans for basing the determination on an image of a user captured by theimage-capturing means.
 21. A non-transitory computer readable mediumencoded with instructions that, when executed, operate to cause acomputer to perform operations comprising: determining, after an objectis displayed in a user interface, a position of a non-control portion ofthe object in the user interface, wherein the object comprises: acontrol portion used for item selection, and the non-control portion;responsive to the determining of the position of the non-control portionof the object in the user interface, defining a guide line at a locationwithin the user interface based on: the determined position of thenon-control portion of the object, and a determined range of motion ofthe control portion of the object; displaying, without overlapping withthe non-control portion of the object, a plurality of items aligned withthe guide line, wherein the determining the position of the non-controlportion of the object in the user interface occurs before the pluralityof items are displayed; highlighting, without moving the guide line, asubset of the plurality of items; receiving a selection of one item ofthe subset; and outputting, based on receiving the selection of one itemof the subset, the selected item.
 22. The computer readable medium ofclaim 21, further comprising instructions for causing the computer tohighlight the subset of the plurality of items based on a location ofthe control portion of the object.
 23. The computer readable medium ofclaim 21, further comprising instructions for causing the computer todetermine the position of the object in the user interface based on animage of a user captured by a camera of a mobile device.